In the field of medical imaging analysis using machine learning, training samples are not enough. In order to solve the problem, a multimodal multi-label transfer learning model was proposed and applied to early diagnosis of Alzheimer's Disease (AD). Specifically, the multimodal multi-label transfer learning model consisted of two components:multi-label transfer learning feature selection and multimodal multi-label learning machine for classification and regression together. Firstly, the multi-label transfer learning feature selection model was built, which was based on the conventional sparse multi-label learning of Lasso (Least absolute shrinkage and selection operator) model for the combination of classification and regression tasks. Secondly, the technique of transfer learning was used to extend the conventional sparse multi-label learning of Lasso model and create the multi-label transfer learning feature selection model that can be performed on training samples from different learning multi-domains. Then, according to the multimodal feature data in the heterogeneous feature space, the multi-kernel learning was used to combine multimodal feature kernel matrix. Finally, the multimodal multi-label learning machine was built, and which was consisted of multi-kernel learning for the combination of multimodal biomarkers and multi-label classification and regression model. To evaluate the effectiveness of the multimodal multi-label transfer learning model, the Alzheimer's Disease Neuroimaging Initiative (ADNI) database was employed. The experimental results on the ADNI database show that the proposed model can recognize Mild Cognitive Impairment Converters (MCI-C) patients from MCI NonConverters (MCI-NC) ones with 79.1% accuracy and predict clinical scores with 0.727 correlation coefficient, so it can significantly improve the performance of early AD diagnosis with the aid of related domain knowledge.